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NETWORK CONGESTION CONTROL 

5 Field Of The Present Invention 

The present invention relates generally to communication networks, and in 
particular to exploiting asynchronous network feedback for source responses executed in a 
source node of a network to control congestion in the network. 

10 Background Of The Present Invention 

Network congestion develops when the datapacket traffic being sent or injected 
exceeds the capacity of the network. Congestion causes the throughput of useful traffic to 
be reduced because each datapacket holds onto network resources longer, or network 
resources are wasted by handling datapackets that are later discarded. 
15 A conventional approach to minimizing network congestion uses closed-loop per- 

flow end-to-end rate control. Congestion feedback is used for updates about network 
congestion to a source of a traffic flow. A Boolean signal provided to the flow source 
indicates whether a set of datapackets injected for the flow experienced any network 
congestion. The source response mechanism adjusts a limit on the rate at which future 
20 datapackets of the flow can be injected into the network in response to each congestion 
feedback signal received for a flow at its source. 

The congestion feedback signal may be implicit or explicit. An implicit signal is 
one detected at the source without support from the network switches. For example, in 
transport control protocol (TCP), an acknowledgment time-out at the source is used to 
25 detect datapacket losses. Such is interpreted as a signal of network congestion, and the 
reception of an acknowledgment (ACK) datapacket with an appropriate sequence number 
is interpreted as a signal indicating no congestion. In contrast, an explicit signal is 
generated by the switches in the network, for example by sending Explicit Congestion 
Notification (ECN) datapackets to flow end-devices. 
30 Some congestion feedback mechanisms favor signaling only the high flow rate 

sources. For example, some mechanisms send congestion flags only when a switch 
datapacket buffer exceeds a certain level of occupancy, and the flows that receive the 
signals are those corresponding to datapackets present in a high occupancy buffer. If the 
number of datapackets that a buffer can store is less than the number of flows that use the 



corresponding link, then only some of the flows sharing the link will have datapackets in 
the buffer when it reaches the threshold occupancy. 

A high rate flow is more likely to have a datapacket in the high occupancy buffer 
than a lower rate flow, because datapackets of the higher rate flow use the buffer more 
frequently than those of a lower rate flow. Therefore, when a switch buffer fills, among all 
the flows sharing the corresponding link, higher rates flow are more likely to receive a 
congestion flag than lower rate flows. Such bias in congestion signaling is strongest in 
network with small buffers that can store only a few datapackets, because then, when a 
buffer becomes highly occupied, the subset of flows represented in the buffer is likely only 
a small fraction of all the flows that share the corresponding network communication link. 

The source response component of congestion control acts at the flow source end- 
device to control the flow's rate limit in response to signals provided by the congestion 
feedback mechanism. When a received feedback signal indicates no congestion, the 
source increases the flow's rate limit. The source receives an ACK datapacket that 
corresponds to one or more of its prior injected datapackets that experienced no congestion 
in the network. The increase is based on an increase function, rnew=finc(r), where "r" is the 
current rate limit, and r„ew is the next rate limit setting for the flow. Similarly, on receipt of 
a congestion flag the source reduces the rate limit based on a decrease function, rnew= 
fdec(r). 

I The rate increase and decrease functions should be designed to operate together to 

enable flows to converge to an operating point that is efficient with high bandwidth 
utilization, and fair, e.g., approximately equal rates to each flow sharing the same 
bottleneck link. 

In networks that use biased congestion feedback mechanisms, a high rate flow is 
5 more likely than any contending lower rate flow to receive a congestion flag each time a 
buffer becomes highly occupied. In such networks, if an update occurs for each 
congestion feedback signal, then higher rate flows perform more frequent rate decrease 
steps than lower rate flows. 

It is an object of the present invention to enable the use of source responses that use 
0 faster increase responses in asynchronous network environments, leading to increased 
network utilization while still achieving convergence to efficient and fair operating points. 
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SUMMARY OF THE PRESENT INVENTION 



Briefly, a network embodiment of the present invention controls congestion by 
monitoring how well packets are actually being received at their respective dataflow 

5 destinations. The destination nodes are outfitted with a monitor that returns an 

acknowledgement (ACK) datapacket to the source node for each reception. The return 
ACK datapackets are marked according to whether congestion was encountered in the 
delivery to the destination. If so, a rate limiter at the source node is signaled to slow down 
the data injection rate. If not, the rate limiter is signaled to dial up the injection rate. 

10 Several dataflows can be independently and simultaneously controlled this way. 

BRIEF DESCRIPTION OF THE DRAWINGS 

15 Fig. 1 is a block diagram of a network embodiment of the present invention; 

Fig. 2 is a flowchart diagram of a rate increase determination process embodiment 
of the present invention; 

Fig. 3 is a flowchart diagram of a congestion flow rate increase/decrease method 
embodiment of the present invention; 
20 Fig. 4 is a flowchart diagram of a rate update method embodiment of the present 

invention; and 

Fig. 5 is a flowchart diagram of a rate update method embodiment of the present 
invention. 

25 

DETAILED DESCRIPTION OF THE EMBODIMENTS 

Fig. 1 represents a network environment 100 in which a source server 102 
communicates with a destination client 104 via a network 106. For example, the network 
30 may include the Internet. Embodiments of the present invention operate in such network 
environments 100. A computer software application 108 running on source server 102 
injects or flows datapackets through a transport layer 1 10 across the network 106 to a 
corresponding transport layer 1 12 on the receiver side. Such traffic ultimately flows to a 



receiving computer software application 1 14. Each source server 102 and destination 
client 104 can simultaneously support multiple such flows. 

Datapackets from the application 108 are passed down through the transport layer 
1 10 of the sender's communication protocol stack. The incoming datapacket is passed 
along to receiving application 1 14 for processing. The rate of datapacket injection into the 
network is controlled by a rate limiter 1 16. An appropriate rate limit is determined and 
enforced for each flow. A datapacket receiver network interface 122 generates an 
acknowledgment (ACK) datapacket automatically each time it receives an incoming 
datapacket. The ACK is returned to source server 102. 
10 A rate limit for each flow is represented at source server 102 by a state variable 

"crt". A separate "crt" state variable is updated for each flow. Each ACK datapacket 
received on path 126 over network 106 is passed to an acknowledgement monitor 128. 
The value of a "crt" state variable 130 is updated and maintained for each corresponding 
flow. The acknowledgement monitor 128 determines whether the relevant flow to which 
15 the ACK datapacket relates is contributing to any congestion in the network 106. 

Such determination can be based on when the ACK datapacket is actually received, 
or based on information coded within it. The acknowledgement monitor 128 may rely on 
explicit congestion notification (ECN) techniques. If congestion is detected, the 
acknowledgement monitor 128 decreases the value of the corresponding flow's rate limit 
20 "crt". Otherwise, the acknowledgement monitor 128 infers from an apparent lack of 
congestion that there is available capacity within the network. It increases the rate limit 
"crt" to allow the rate limiter 1 16 to set a higher rate of injection into the network for the 
relevant flow. 

The operation of the acknowledgement monitor 128 is particularly suited to 
25 application to the INFINIBAND standard. The INFINIBAND Architecture is an industry 
standard, channel-based, switched fabric, interconnect architecture for servers. 
INFINIBAND architecture dictates a new way servers are to be buih, deployed, and 

managed. 

In any network environment, a primary goal is to minimize or avoid network 
30 congestion and achieve fairness for contending flows. Such results in higher network 
utilization than was previous possible. 

Embodiments of the present invention allow source responses in a network source 
node, e.g., source server 102, to increase and decrease flow rate (fi„c(r) and fdec(r)), to 
exploit congestion signaling bias in order to improve performance. 



A method embodiment of the present invention provides for controlling a plurality 
of datapacket flows into a network. Such is based on an asynchronous rate control 
procedure, receiving network congestion feedback, and adjusting a datapacket injection 
rate of each of the datapacket flows based on the congestion feedback. The datapacket 

5 injection rates are adjusted according to a rate increase function when the congestion 

feedback indicates no congestion. Following any decrease in the datapacket injection rate, 
the datapacket injection rate is incremented. The period set for such increases is at least 
the inverse of the minimum injection rate. 

Method embodiments of the present invention provide network source node 

10 responses which converge to fair and efficient networks operating points. Source 

responses include fast increase multiplicative decrease (FIMD), and linear inter-datapacket 
delay (LIPD). 

Conventional designs provide convergence under the synchronous update 
assumption by using conservative responses to congestion feedback signals. Embodiments 
15 of the present invention exploit the datapacket marking bias of asynchronous updates to 
weaken the conditions for fairness convergence. The weakened conditions enable the use 
of responses that do not improve fairness in a synchronous scenario. The FIMD and LIPD 
source responses reclaim bandwidth more rapidly than responses that have the same 
decrease behavior but satisfy the stronger conditions for rate increase. Quicker 
20 reclamation of bandwidth using the new functions yields higher network bandwidth and 
throughput, especially in dynamic environments in which flows come and go. 

In the example congestion feedback mechanism, the network uses datapacket 
marking to provide a congestion flag. Such is a form of forward explicit congestion 
notification (FECN). Whenever a switch buffer reaches a state of high occupancy, all the 
25 datapackets within the buffer are marked. The datapacket marks are returned by a flow's 
destination node to the flow's source node by marked ACK datapackets. A signal 
indicating no congestion is communicated to the flow's source node by unmarked ACK 
datapackets. 

The source response mechanism controls the injection of datapackets into the 
30 network based on the information provided by the congestion feedback mechanism. On 
receipt of a congestion feedback signal indicating no congestion, the source increases the 
flow's rate limit based on an increase function, r„ew= finc(r), where "r" is the current rate 
limit, and r„ew is the next rate limit setting for the flow. The source reduces the rate limit 
based on a decrease function, r„ew= fdec(r) on receipt of a feedback congestion flag,. 
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In the absence of marks, it is desirable for the rate to gradually increase over time. 
Let F'i„c(t), for t>0, be a family of continuous monotonic increasing functions, each of 
which describes the desired flow rate increase behavior as a function of time since the last 
rate decrease to an arbitrary rate F^i„c(0)=r (Rmin < "r" < Rmax, where R„,i„ and are the 

5 minimum and maximum values for a flow rate in datapackets/sec, respectively. Rmin and 
Rn,ax are constants for a particular network and implementation. The value of will 
typically equal the full bandwidth of the network link (e.g., if the link is ten gigabits per 
second, Rmax will also be ten gigabits per seconds). Rn,in can be any value that is less than 
Rmax and greater than zero and it refers to the lowest rate limit that the implementation of 

10 the sending node can support. Determination of R^in and Rn,ax is based on the notion that 
the rate control implementation can support only a finite set of rates, for example because 
the rate is represented digitally using a finite-width register. R^in is the smallest value in 
the finite set of supported rates. The particular values of R^ax and R^n for a network rate 
control implementation will be known or readily determined by the network designer and 

15 can be factored into the design of the source responses. Since the increase function fi„c(r) 
is defined as a function of the current rate, the time behavior of the rate increase should be 
independent of the past history of the flow rate, e.g., it should be independent of the 
elapsed time since the last decrease. Therefore, the time behavior of the rate for two 
arbitrary initial rates r, and rj, (Rmin < r, < rj < Rmax), should be identical for rates "r" > r2, 

20 i.e.: 

F'^nc(t) = F'S„c(t+t') 

for t>0, and t' such that [1] 
F'S„e(t') = r2. 

25 It follows that the rate increase behavior can be represented by just one member of 

the family of functions: Finc(t) = F^™'"inc(t). All other functions F'i„e(t), for R™„ < "r" < 
Rmax, can be obtained by shifting the time origin of Finc(t), as described in Equation [1]. 

A recovery time/time duration Trec(r) is defined for a flow at rate "r" as the time 
elapsed from the time the flow rate is decreased from rate "r", due to a marked ACK, until 

30 the time the flow rate recovers to its original rate "r", assuming no other marked ACK is 
received until rate "r" is achieved. 

If the recovery time or time duration of a lower rate flow is longer than that of a 
higher rate flow, flow rates may diverge and the higher rate flow may take over the entire 
bottleneck bandwidth, creating an unfair operating point. To avoid this situation and 
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promote fair allocation of bandwidth, source responses must satisfy the relaxed 

convergence requirement, 

Trec(ri) = Trec(r2) =Trec = 1/Rmin (R'min<ri<Rmax, R'min<r2<Rmax) , [2] 
where fdec(R'nun) = Rmin • 

5 The recovery time is a constant 1/Rmin for all rates higher than R min- Such is the 

highest rate from which a single decrease step assigns to a flow the minimum rate Rmin- In 
the case of a synchronous network feedback for rate decrease, the result of satisfying the 
property in [2] is that any two flows, with rates ri and T2 before the rate decrease, will 
recover to their original rates ri and T2 at the same time. Therefore, after the recovery, 
10 fairness is not decreased but only maintained. While the Chiu and Jain, and the Bansal and 
Balakrishnan conditions require that fairness be strictly improved in a sequence of 
decrease/increase phases assuming a synchronous feedback scenario, the presently 
formulated condition requires only that fairness be maintained in the same scenario. 
The choice of 1/Rn,in for Tree follows from the following argument. At the 
15 minimum rate Rmm. the interval of time between two consecutively transmitted datapackets 
is 1/Rmin- Thus the expected time interval between the reception of two consecutive ACK's 
is also 1/Rmin. Therefore, assuming a marked ACK causes the rate to be decreased from 
rate R'mi„ to the minimum rate Rmin, the next rate change can only occur when the next 
ACK is received, i.e. after an expected time l/Rmin- Therefore the minimum possible 
20 recovery time for rate R'min is 1/Rmin (assuming the magnitude of an increase step cannot 
exceed the magnitude of a decrease step). Since the same recovery time is desired for any 
rate "r", 1/Rmin is the minimum possible recovery time for any rate "r", R'min ^ "r" ^ Rmax- 
In order to reclaim unused bandwidth as fast as possible, 1/Rmin is chosen as this minimum 
value for the recovery time Tree (r) for any rate "r", R min< "r" <Rmax- 
25 In order to achieve relaxed convergence requirement, the time behavior of a flow 

rate Finc(t) should satisfy the following condition (difference equation): 

Finc(t) = fdcc(Fmc(t+Trec)), Or [3] 
Fme(t) = fdec(Fmc(t+ 1/Rmin)) 

This condition requires that after a decrease event, the increase function recovers 
30 the rate limit back to the particular rate prevalent prior to the decrease event in time 1/Rmin- 
Such is the constant recovery time Tree from relaxed convergence requirement. Equation 
[2]. 

Fig. 2 represents a rate increase determination process embodiment of the present 
invention, and is referred to herein by the general reference numeral 200. Given an 



arbitrary decrease function fdec(r), process 200 determines fi„c(r) such that relaxed 
convergence requirement is satisfied. In a step 202, a continuous monotonically increasing 
function F.nc(t) is found that is a solution for the difference equation described in [3]. Such 
can be done using conventional techniques for solving difference-differential equations. 
Then all other functions F'i„c(t) are obtained, for an arbitrary rate "r", shifting the time 
origin of Fi„c(t). according to Equation [1]. At step 204, these functions are used to 
generate a look-up table for all possible settings of "crt". Given Fi„c(t), the increase 
function finc(r) is obtained at step 206. At a given rate "r", the expected interval between 
consecutive datapackets, and thus between consecutive ACK datapackets is 1/r. Assuming 
that at the previous adjustment the flow rate is set to "r", the next rate adjustment will 
occur at the reception of the next ACK, e.g., after an expected time 1/r. Thus, finc(r) = 
F'inc(l/r)> with a ceiling of 



fi„c(r) =min ( F^„c(l/r), Rmax) ■ [4] 



15 



In order that the increase function does not cause the injection rate to exceed the 
maximum injection rate R^ax, the lesser of the newly calculated increased rate and is 
chosen as the new rate. 

At source server 102, the acknowledgement monitor 128 implements the rate 
20 increase and decrease functions. To ensure rapid response to changes in network 

conditions, it is important that the implementation of this logic module be fast enough to 
adjust the value of the rate limit "crt" each time an ACK datapacket arrives from the 
network 106. In a high speed network, the time between consecutive ACK datapacket 
arrivals at a source node may be very short, for example in the order of a few tens or 
25 hundreds of nanoseconds. Since the decrease and increase functions may be complex 
mathematical expressions involving time-consuming computational operations such as 
floating point division or exponentiation, there may not be sufficient time to calculate the 
function outputs unless specialized and expensive hardware is provided. As a less costly 
alternative, the output of each function can be pre-computed for all possible settings of 
30 "crt" and these outputs can be stored in a memory look-up table indexed according to "crt" 
value. 

During operation, the acknowledgement monitor 128 can then determine the 
correct rate adjustment and corresponding new "crt" value by performing a fast access to 
the appropriate look-up table in memory. Such look-up table is effectively part of the 
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functional block of the acknowledgement monitor 128, although it may not have exactly 
the same location in hardware. 

The fast increase multiplicative decrease (FIMD) source response function a 
multiplicative rate decrease function is adopted. Such is the same decrease function used 
5 by the traditional additive increase multiplicative decrease (AIMD) function, 
fdec(r) = Max (r/m, Rmin). where m > 1 is constant 

From Equation [3], Fi„c(t) must satisfy, Fi„c(t+Trec) = m * Fi„c(t). 
A continuous monotonically increasing function that satisfies this condition with, 
Fi„c(0) = Rmin, is Fi„c(t) = Rmin* m . For any rate "r", R™„<r<Rmax, there exists a t'for 
10 which "r" = FincCf) = Rmin * m '""^^ Therefore, 

F'i„e(t) = F.e(t-Ht') = Rmin * m ^'^^ * m = ' V * m 

and 

fi„,(r) = Min(F'mc(l/r), Rmax) = Min(r*m ' Rmax)= Min(r*m«-'^ , Rmax) • 

15 Fig. 3 represents a procedure 300 used by an FIMD source response mechanism. A 

flow source end-device or node increases and decreases a flow's rate limit in response to 
congestion feedback. Procedure 300 reads and writes the value of a state variable "crt". 
Such records the current rate limit setting for a flow. In a hardware implementation of the 
source response mechanism, the "crt" variable may be implemented as a hardware register. 

20 Another procedure at the source end-device reads the value of "crt" and prevents 

datapackets for the flow from being injected into the network at a higher rate than the 
value of crt specifies. In a step 304, an ACK datapacket just received is checked to see 
marked. 

A marked ACK datapacket indicates congestion, so the source reduces the rate 
25 limit "crt" by a constant, m > 1 at step 306. In a step 308, an updated rate "r" is then 

checked against the minimum injection rate Rmi„. If the new rate is less than Rmin, "crt" is 
assigned the value of Rmin at step 3 10. If the new rate is not less than Rmin, "crt" assigned 
the value of the new rate at step 312. 

If an unmarked ACK is received, the source increases the rate limit "crt" by a 
30 constant, m''"^"'"' , at step 314. The increased rate is then compared against the maximum 
injection rate, Rmax, at step 316. If the new rate is greater than Rmax, "crt" is assigned the 
value of Rmax at step 318. If the new rate is not greater than Rmax, "crt" is assigned the 
value of the new rate at step 320. 
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Once the state variable "crt" is assigned the appropriate new rate according to the 
above steps, the update procedure is ended at step 322 and awaits receipt of a new ACK 
datapacket at 302 to repeat the update procedure 300. 

The LIPD response function is based on a decrease function that increases inter- 
5 datapacket delay (IPD) linearly. IPD is the idle period length that is inserted between the 
injections of consecutive datapackets of a flow, expressed in units of datapacket 
transmission time. A flow operating at an IPD of "ipd" corresponds to a flow rate of 
Rmax/(l+"ipd")- A flow's rate decrease is defined as an increment by one of the flow's IPD 
value (which increases the inter-datapacket delay by one datapacket transmission time). 
10 Such rate decrease function is intuitively attractive for the following reason. If "n" 

identical flows share a bottleneck link, the optimal rate for each flow is Rmax/n , where the 
IPD is equal to n-1. 

If a new flow is introduced to a link that already has "n" flows that are operating at 
optimal rate, than one datapacket from each of the "n" original flows receives a mark and 

15 each one has its rate limit reduced from Rmax /n to the new value Rmax/(n+l). Such 

becomes the new optimal rate limit for these flows. With a decrease function based on 
incrementing the IPD by one, when a new flow is introduced, the flow rates of the 
previously resident flows converge in one decrease step to the new optimal rate value. 
This, instead of oscillating and slowly converging to the new optimal rate value, as they 

20 would with conventional decrease functions. Also, at lower rates this function decreases 
the rate by smaller steps than a multiplicative decrease function, e.g., FIMD and AIMD. 

When several dynamic flows share a link, smaller decrease steps lower oscillation 
amplitude and improve overall link utilization. The rate decrease function can be derived 
using the inverse relationship of flow rate to the flow IPD, e.g., 

25 

fdec(r) = Max ( Rmax /(I + Rmax/r), Rmin ) • 

From Equation [2], Finc(t) must satisfy, 

30 Fi„c(t+T^) = Rmax /( Rmax/Finc(t) - D • 

A continuous monotonically increasing function that satisfies this condition with 

Finc(O) = Rmin, iS, 

Finc(t) = Rmax / (Rmax/Rmin — t/Trec) • 



For any rate "r", Rmin^^Rmax, there exists a t' for which r= Finc(t')= Rmax/CRmax/Rmin 

-t'/Trec)- Therefore, 

F^nc(t) = Fi„c(t+t') = Rmax / (Rmax/Rn.n - t'/T^c- tA^rec) 
= Rmax / (Rmax/ Fi„c(t') - tfT^) = Rmax / (Rmax/ "f" " tn^rec ) 

and, 

finc(r) = Min(F'i„c(l/r), Rmax) 
= Min( R„^ I (Rmax/ "r" - l/(r*T^) ), Rmax) 
= Min( Rmax / (Rmax/ "r" - Rmin/r), Rmax) 
= Min( "r" / ( 1 - Rmin/ Rmax), Rmax) • 

Fig. 4 represents a procedure 400 used by a LIPD source response mechanism at a 
flow source end-device (node) to increase and decrease a flow's rate limit in response to 
congestion feedback. Such is similar in nature to update procedure 300 (Fig. 3). 
Procedure 400 reads and writes the value of a state variable "crt". Such records the current 
rate limit setting for the flow. Such description assumes the existence of another procedure 
at the source end-device that reads the value of "crt" and prevents datapackets for the flow 
from being injected into the network at a higher rate than the value of "crt" specifies. At 
step 402 an ACK datapacket is received and, at step 404, it is checked to determine 
whether it is marked. If it is a marked ACK datapacket, indicating congestion, the source 
reduces the rate limit "crt", at step 406, to a value that corresponds to an increase of the 
inter-datapacket delay by one unit of datapacket transmission time l/Rmax- The updated 
rate "r" is then checked against the minimum injection rate R^in at step 408. If the new 
rate is less than R^in, "crt" is assigned the value of R„,in at step 410. If the new rate is not 
less than Rmi„, "crt" assigned the value of the new rate at step 412. 

If it is an unmarked ACK datapacket, the source increases the rate limit "crt" at step 
414. The new value decreases of inter-datapacket delay by a fraction Rmin/ "r" of one 
datapacket transmission time l/Rmax- The increased rate is then compared against the 
maximum injection rate, Rmax, at step 416. If the new rate is greater than Rmax, "crt" is 
assigned the value of Rmax at step 418. In a step 420, if the new rate is not greater than 
Rmax, "crt" is assigned the value of the new rate. 

Fig. 5 represents how the LIPD source response mechanism could be deployed in 
an INFINIBAND, or other network. The network controls the rate limit for a flow by 
using an integer state variable "ipd". Such specifies the number of idle datapacket 
transmission times to insert before transmitting each datapacket. Such description 
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describes how to set the value of "ipd", but it is assumed that another procedure at the 
source end-device 102 is operating that reads the value of "ipd" and inserts the delay that it 
specifies for the flow after each of the datapackets is injected into the network. The LIPD 
procedure applied to INFINfflAND is similar to that described in relation to Fig. 4, except 
5 for extra steps (522 to 528) at the end. Such set "ipd" to an integer value that yields a rate 
that is closest to the ideal rate limit. Such is represented by variable "crt". 

Fig. 5 represents a rate update procedure 500 for the LIPD source response function 
implemented on a network conforming to the INFINffi AND standard. Steps 502 to 520 
are similar to steps 402 to 420 (Fig. 4). After a step 522, the inter-datapacket delay, "ipd", 
10 corresponding to rate "crt" is computed and rounded down to the closest integer value. 
The maximum injection rate, R^ax, is divided by "crt" and then subtracting one. 

The rate of a flow can be mathematically represented as Rmax /("ipd" +1). In a step 
524, the error in the flow rate due to the rounding down of "ipd" is calculated, given by 
(Rmax /("ipd" + 1) - "crt"). Such error value from the rounding down of "ipd" is compared 
15 with the error in the rate which would occur if the "ipd" were rounded up (given by "crt" - 
Rmax/("ipd"+2)). Thus if the error from rounding down the "ipd" is greater than that from 
rounding it up. the "ipd" is incremented by one at step 526. Otherwise, the rounded down 
"ipd" is considered to be correct, and the update procedure 500 finishes at step 528. Steps 
522-528 ensure the inter-datapacket delay is an integer value. 
20 In one embodiment, the present invention provides a method for controlling a 

plurality of datapacket flows into a network based on asynchronous rate control procedure. 
The method comprises receiving network congestion feedback and adjusting a datapacket 
injection rate of each of the datapacket flows based on the congestion feedback. The 
adjusting comprises increasing the datapacket injection rate according to a rate increase 
25 function if the congestion feedback indicates no congestion. Following any decrease of the 
datapacket injection rate, the datapacket injection rate is increased to the particular rate in a 
time duration which is at least the inverse of a predetermined minimum injection rate. 

Although the present invention has been described in terms of the presently 
preferred embodiments, it is to be understood that the disclosure is not to be interpreted as 
30 limiting. Various alterations and modifications will no doubt become apparent to those 

skilled in the art after having read the above disclosure. Accordingly, it is intended that the 
appended claims be interpreted as covering all alterations and modifications as fall within 
the true spirit and scope of the present invention. 
What is claimed is: 



